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Abstract. As people attach increasing im- 
portance to the learning environment and 
design of teaching materials in STEM, how 
to design teaching materials that enable 
students to seamlessly integrate the learn- 
ing of scientific investigation, engineering 
design, mathematical reasoning, and 
technical skills has become increasingly 
important. The purpose of this research 
was to explore the behavioral intention 

of GAR-STEM teaching application users. 
The results showed that the best predic- 
tor of practicability, entertainment, and 
media interactivity for intention towards 
GAR-STEM app design was usage attitude 
(R?= 0.547), followed by intent to use 

(R? = 0.528), practicability (R? = 0.408), and 
entertainment (R? = 0.186). The proposed 
model explained 52.8 % of the variance in 
behavioral intention. The overall findings 
suggested that usage attitude and intent 
to use may augment its function as a key 
factor for the procedural intention toward 
GAR-STEM app design. 
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Introduction 


With the rapid development of digital media in recent years, many 
textbooks have been equipped with the features of entertainment-oriented 
media. The gamification design idea has been integrated with new scientific 
technologies to facilitate teaching. In this way, the application of games in 
digital learning has been increased and the creative application of learning 
materials has been restarted. In particular, the application of augmented 
reality gamification in STEM education has been reinforced with each pass- 
ing day. As acombination of science (S), technology (T), engineering (E), and 
mathematics (M), STEM covers the experiences in daily life and applications 
in the workplace. All these subjects (STEM) are interlinked and connected 
with the complicated problems facing individuals every day (Talley, 2016). 
The integrated knowledge education of STEM offers better solutions to tackle 
the challenges in daily life and in the workplace (Webb, 2013). Therefore, it 
is necessary for students in grades K-12 and college students to understand 
the concepts of STEM and develop the ability to integrate and apply these 
subjects so that they will have more chances to put STEM knowledge into 
practice (Bybee, 2013; English, 2017). Science, technology, engineering, and 
mathematics are mutually independent, and the core concepts of each sub- 
ject provide a solid foundation for further studies. As people pay increasing 
attention to the application of STEM in K-12 and college education, inter- 
disciplinary integration can help students gain a better understanding of 
subject concepts and transfer knowledge from one subject to another (Bell, 
2016). According to the results of relevant studies, the combination of creative 
and advanced technologies and teaching can enhance the STEM learning 
participation of learners and improve their learning attitude towards STEM. 
Moreover, a simulated learning environment can strengthen the cognition 
and motivation of learning and make learning more effective (Kim et al., 
2015; Stull, Fiorella, Gainer, & Mayer, 2018). The integration of simulated STEM 
learning environments is very important. Therefore, how to help students 
understand scientific concepts and solve interdisciplinary problems through 
creatively-simulated and seamless experience-based learning environments 
is an essential topic and one of the challenges facing education researchers 
(Katehi, Pearson, & Feder, 2009). 
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Augmented reality (AR) is an information and communication technology with scientific applications and 
has become a hot issue in scientific and education studies (Akgayir & Akgayir, 2017). With the help of scientific 
technologies, AR incorporates virtual 2D or 3D computer simulations into a real environment (Lee, 2012). Since its 
introduction, the technology has been widely applied in education. For instance, it has been used in the teaching 
of the tissues and organs inside the human body and that of earth science. The knowledge of different subjects 
has been incorporated into AR recognition labels on textbooks. The scanning of these labels leads to the display of 
3D objects. Moreover, AR has been utilized to explore the models and performance of learning behavior, and it has 
been shown that AR has the potential of being applied to teaching (Azuma et al., 2001; Lee, 2012). AR-supported 
teaching features sensory organ immersion, simple operation, and alow cost, which together contribute to a greater 
learning effect (Cheng &Tsai, 2013). Relevant researches showed that the application of AR to science, technology, 
engineering, and mathematics (STEM) has potential educational value, including spatial understanding, technol- 
ogy application, problem-solving ability, concept understanding, and scientific exploration and learning (Cheng 
& Tsai, 2013; Wu, Lee, Chang, & Liang, 2013). Especially, AR is most welcomed when it is applied to such fields as 
the humanities, art, and science (Liu & Tsai, 2013; Martin-Gutiérrez et al., 2010; Squire & Jan, 2007). A previous re- 
search found that when AR-supporting teaching was applied in a physics lab for college students, it enhanced the 
learning efficacy of the students and equipped them with a positive attitude towards their work (Akcayir, Pektas, 
& Ocak, 2016). Some studies have revealed that AR has many advantages in an educational setting (Cheng & Tsai, 
2013). For example, AR helps students with the exploration of the real world (Dede, 2009). By combining virtual 
elements with real objects, AR enables individuals to observe items that cannot be observed with the naked eye 
(Wu, Lee, Chang, & Liang, 2013). Hence, it can strengthen students’ learning motivation and help them develop a 
stronger ability to explore things (Sotiriou & Bogner, 2008). 

Gamification is defined as the usage of game mechanisms in daily activities. In 2010, there was a new trend 
of gamification, in which game-based projects and game features were used in ways unrelated to a game (Radoff, 
2012; Smith, 2012; Wu, 2012). Gamification refers to game elements being applied to a non-game environment 
(Deterding, Dixon, Khaled, & Nacke, 2011). Aside from enriching the joyous experience in services and increasing 
the intention to use, it can create interaction among groups (Hamari & Koivisto, 2015; Huotari & Hamari, 2017; 
Liu, Santhanam, & Webster, 2017) and promote usability, productivity, and satisfaction (Saha, Manna, & Geetha, 
2012; Rajanen & Rajanen, 2017), thus pushing forward publicity and generating a positive effect on performance 
(Rodrigues, Costa, & Oliveira, 2014). In recent years, gamification has been increasingly applied to various subjects, 
including education (Simdes, Redondo, & Vilas, 2013; Kim, Song, Lockee, & Burton, 2018), environmental and ecologi- 
cal behavior (Prestopnik & Tang, 2015), health and medicine (Fleming et al., 2017), creative business and marketing 
(Roth, Schneckenberg, & Tsai, 2015), software development and machine learning (Chow & Huang, 2017), politics 
(Santos et al., 2015), tourism (Saoud & Jung, 2018), and energy popularization and application (Nicholson, 2012). 
Gamification is designed to adapt enjoyable and interesting game elements to the activities being publicized. 
Applying these elements to teaching strengthens learning motivation (Su & Cheng, 2015; Su, 2017; Su, 2018). 
The application of gamification to social activities is a highly pronounced category, especially in social network 
websites where people share their daily lives. Social games and gamification are also common in other fields, such 
as measurements of user loyalty, achievement (such as scores, virtual currencies, and levels) and employment 
(Zichermann, 2011). Therefore, the gamification of education refers to game elements being applied to learning 
environments, and it is often supported by ICT (information and communication technology). 


Relation between Media Interactivity, Entertainment, and Practicability 


The Media Richness Theory (Daft & Lengel, 1986) refers to the capability to carry information in media. Since 
different types of media load various information capabilities, it causes differences in how users select media. Daft, 
Lengel and Trevino (1987) divided media richness into the concepts of: (1) instant feedback; (2) multiple cues; (3) 
personal focus; and (4) language variety. Rice (1993) indicated that different types of media present distinct char- 
acteristics and result in various communication effects because of different objectives. Dennis and Kinney (1998) 
mentioned that using computers as the communication media could provide instantaneous feedback, deliver 
multiple cues, communicate with diverse languages, and focalize individuals. Wang, Hsieh, and Song (2012) pointed 
out the positive effect of users’ perceived media richness on the perceived entertainment. Sullivan (1999) explained 
that interactivity is the dialogue between information systems and users; in addition to the interaction, informa- 
tion system users can respond to the information and content offered by the information systems in the process. 
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Previous literature (Coyle & Thorson, 2001; Lan & Sie, 2010; Lu et al., 2014; Peltokorpi, 2015; Zhao, Hen, & Wang, 
2016) regarded the positive effect of interactivity on the practicability of information systems and the assistance in 
better consistency between attitude towards use and use behavior. Thus, the following hypotheses were proposed: 

H1: Media interactivity presents a positive effect on entertainment. 

H2: Media interactivity shows a positive effect on attitude towards use. 

H3: Media interactivity reveals a positive effect on practicability. 


Relation between Entertainment Attitude and Behavioral Intention 


The Theory of Planned Behavior states that behavior is controlled by behavioral intention and that other 
possible factors might indirectly influence behavior through behavioral intention, which refers to an individual's 
intention to engage in certain behavior. Behavioral attitude, referring to the perceived positive or negative evalu- 
ations of an individual executing certain behavior, is evaluated by behavior belief and results. An individual with 
a more positive attitude towards a behavior will present higher behavioral intention, and vice versa. Subjective 
norms refer to an influential or important person's opinions about specific behavior, which are determined by 
the beliefs and motivation of norms. Ajzen and Fishbein (1977) argued that attitudes and subjective norms are 
the major factors in behavioral intention. From past research, The Theory of Planned Behavior can be applied to 
explain different types of behavior. In the process of enjoyment, elements related to imagination, feelings, fun, and 
symbol meanings often appear in entertainment (Hirschman & Holbrook, 1982), which could be festive or ludic 
perceptions (Sherry, 1990). The characteristics of entertainment have been applied to research on behavior (Lee, 
2009; Maes et al., 2014; Scarpi, 2005; Sherry, 1990; Wu & Chen, 2005) and present subjective and individual contents 
(Babin, Darden, & Griffin, 1994). From the aspect of using a system, users perceiving the higher “entertainment” of 
a decision system will show a higher intention of use (Gopal, Bostrom, & Chin, 1992). Accordingly, the following 
hypotheses were proposed in this research: 

H4: Entertainment reveals a positive effect on attitude. 
H5: Entertainment presents a positive effect on behavioral intention. 


Relation between Practicability, Attitude, and Behavioral Intention 


The Technology Acceptance Model proposed by Davis (1989) is an extension of the Theory of Reasoned Action, 
which aims to predict and understand human behavior. It is considered that individual behavior is determined by 
behavioral intention, while behavioral intention is possibly affected by personal attitudes and subjective norms. Ac- 
cording to Davis’(1989) Technology Acceptance Model, user attitudes towards a system are influenced by perceived 
ease of use and perceived usefulness, users’ behavioral intention is affected by attitude and perceived usefulness, 
and perceived usefulness affects the perceived ease of use. This model has been broadly applied to various studies 
on technology acceptance behavior, and the hypotheses in the model have been verified numerous times (Ahn, 
Ryu, & Han, 2007; Al-Gahtani, 2016; Mohammadi, 2015; Moon & Kim, 2001; Sanchez & Hueros, 2010; Venkatesh & 
Bala, 2008). The following hypotheses in the Technology Acceptance Model were therefore included in this research: 

H6: Practicability presents a positive effect on attitude. 
H7: Practicability shows a positive effect on behavioral intention. 
H8: Attitude towards use reveals a positive effect on behavioral intention. 


Therefore, to fill the gap left by the literature, this research adopted gamification augmented reality (GAR- 
STEM) and the Media Richness Theory to explore usage behavior. In addition, IPMA was utilized to analyze the 
features of GAR-STEM design, so as to find out its advantages and limitations. Matters worthy of attention and 
advice improving the intention to use GAR-STEM in the future were also included. This research proposed the fol- 
lowing three research goals: 

1) To integrate the explanation power of the GAR-STEM app user intention model; 

2) To understand GAR-STEM app user intention; 

3) To integrate the analysis strategies of Importance-Performance Map Analysis (IPMA) for the matrix 
analysis of the importance and performance of GAR-STEM app user intention in order to offer GAR- 
STEM apps designers with better strategic design thinking. 
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Research Methodology 


Partial Least Squares (PLS) is a form of component-based Structural Equation Modeling (SEM) that can deal with 
smaller samples (30-100) and predict non-normal data with more variables, as well as deal with reflective indicators 
and formative indicators, thus making it suitable for developing theories (Chin & Newsted, 1999; Gefen, Straub, & 
Boudreau, 2000). Furthermore, PLS can overcome multicollinearity problems, effectively deal with moderating data 
and missing values, and present favorable prediction and explanation capabilities. PLS was adopted in this research 
because the retrieved data did not completely conform to a normal distribution and because only 177 samples 
were retrieved. In addition, the research issue was about theoretical exploration and discovery. As the number of 
research samples was small, analysis with PLS presented favorable prediction and explanation capabilities. Such 
a small sample would also not be affected by the number of samples and the distribution of variables. For the 
stable estimation of variables, 500 iterations of bootstrap resampling has been suggested for testing (Chin, 1998). 


Research Structure 


The integration of practicability, entertainment, and media interactivity was proposed to construct the mea- 
surement model for the intention of using GAR-STEM apps. After establishing the hypotheses, the positive, direct, 
and significant effect of media interactivity on entertainment, attitude towards usage, and behavioral intention 
was as shown in Figure 1. 
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Figure 1. Model of behavioral intention to use GAR-STEM apps. 


Operational Definition of Constructs 


In order to test the above research hypotheses, a conceptual model was utilized, and the corresponding 
questions were constructed in this research. According to the conceptual model, the research variables in this 
research included media interactivity, entertainment, attitude towards use, practicability, and behavioral intention. 
A structured questionnaire was presented to the participants and was scored using a Likert 7-point scale (with an- 
swers ranging from extremely disagree to extremely agree) to collect relevant data. The results were processed and 
analyzed using SPSS, and PLS was utilized to analyze the research model. The five latent variables in this research 
were reflective indicators used for analyzing the relevance between variables and indicators. The definitions and 
measurement of the variables, including the five dimensions of media interactivity, entertainment, practicability, 
attitude towards use, and behavioral intention, as well as 16 questions, are listed in Table 1. 
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Table 1. |§ Operational definitions of constructs. 

Variable Operational definition Questionnaire item Beerente 
The visual design of the app is attractive. 
- Overall speaking, the design of the app’s user interface is 
Interactivity between app favorable. 
Media interactivity users and the app in the usage (Coyle & Thorson, 2001) 


process (MI) Good interactivity with other users on the app. 


Being able to establish friendships with other users through 
the app. 


BD canedieniccaminontor The app offers several funny and interesting functions. 
Entertainment app users about the functions | am pleased at using the app’s functions. 


offered by the app (EN) 


(Csikszentmihalyi, 1975; 
Moon & Kim, 2001) 
| perceive fun in using the app. 


The app offers the contents | desire. 


App users’ perception of the Overall speaking, the app is useful. (Chen, Clifford, & Wells, 
Practicability information or services offered . noi 2002; Chen & Wells, 
by the app (PR) The operation of the app is simple. 1999: Ducoffe, 1996) 


The operation of the app is clear and understandable. 


App users’ perceived preference Overall speaking, | like to use the app. 


Attitude towards use 
for the app (ATT) | have a positive evaluation about the app. 


(Davis, 1989) 
| will continuously use the app. 
App users’ evaluation of the 
Behavioral intention —_ intention and possibility to 
continuously use the app (BI) 


| will still choose to use the app even though there are other 


ie (Davis, 1989) 
similar ones. 


| will recommend the app to others. 


Sample and Scale Analysis 


(1) Formal questionnaire respondents 

In the formal test, in consideration of the impact of regional disparities, a two-step sampling method was 
applied in this research, namely, the stratified cluster sampling method and quota sampling. In step 1, the strati- 
fied cluster sampling method was applied based on school size in consideration of regional disparities, and four 
schools were selected as samples. In step 2, quota sampling was applied, and 186 samples were selected from the 
sample schools by considering different sampling percentages for different grades and dispatch questionnaires 
were dispatched. A total of 177 valid questionnaires were recovered, resulting in an availability of 95.1%. As the 
late responses could be similar to those who did not respond, the early response (n = 106) and late response (n = 
71) samples were tested for nonresponse bias (Armstrong & Overton, 1977). All measured variables and control 
variables of the two groups of samples were compared with a t-test to judge the possible bias of the late responses. 
The result showed no significant difference in the responses between the first and the second stage, revealing 
insignificant nonresponse bias in this research. 


(2) Test of Common Method Variance 

Common Method Variance (CMV) refers to the man-made covariant relevance between independent vari- 
ables and dependent variables resulted from using the same data sources, participants, measuring environments, 
scale semantics, and scale characteristics. CMV can be used to test if a single factor is able to explain more than 
50% of the variance of all variables in the initial solution during factor analysis (Podsakoff, 2003). For this reason, 
Harman's One-Factor Test was applied to test for CMV among the research variables in this research and factor 
analysis was used to analyze the factors extracted from the initial solution in order to test for Common Method 
Variance. A total of nine factors with eigenvalues larger than 1 were extracted using confirmatory factor analysis. 
The explained variance of the first factor was 24.25% (maximum), which was less than 50%, and the accumulated 
explained variance of the nine factors was 83.22%. Accordingly, there was no obvious common source bias in this 
research (Saraf, Langdon, & Gosain, 2007). 
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Research Results 


In this section, the research results are presented in three stages. The reliability and validity of the measurement 
model were analyzed during the first stage, the path coefficient test of the structural model and the estimation of 
the model prediction indicator were analyzed during the second stage, and the Importance-Performance Matrix 
Analysis (IPMA) of the structural model was performed during the third stage. The detailed steps and explanations 
are described in the following sections. 


Reliability and Validity Analysis of the Measurement Model 


According to Fornell and Larcker’s (1981) principles of convergent validity, the test of convergent validity 
of the designed questions needed to satisfy the following conditions. First, the factor loading (A) should be sig- 
nificant and higher than 0.5 (Chin, 1998). Second, the composite reliability (CR) should be larger than 0.6 (Chin, 
1998). Third, the discriminant validity-average variance extracted (AVE) should be larger than 0.5 (Chin, 1998) and 
the cross-loading matrix (Chin, 1998). The reliability analysis of the dimensions in this research was organized as 
shown in Table 2. The factor loadings showed a significance of p=.001 and all questions were higher than 0.5, the 
composite reliability was 0.80-0.96 and larger than 0.8, and the average variance extracted was between 0.67 and 
0.85 and above 0.5, thus satisfying the above conditions. The measurement model showed that the factor loadings 
of all questions were 0.75-0.96 and the p value reached the significance level of 0.05, thus indicating convergent 
validity (Chin, 1998). The estimation of discriminant validity was tested from two directions (Henseler, Ringle, & 
Sinkovics, 2009). One was the cross-loading matrix, which indicated that the individual question loadings of the 
variables in this research were higher than the loadings of other variables (Fornell & Larcker, 1981). The other was 
the Fornell-Larcker criterion; that is, the AVE of each variable needed to be larger than the square of the correlation 
coefficient between paired variables to show the discriminant validity of the research variables (Fornell & Larcker, 
1981; Segars & Grover, 1998). Table 2 presents the average variance extracted square roots of all variables, which 
were larger than the correlation value between variables. Table 3 shows that the individual question loadings of 
the variables were higher than those of other variables. Accordingly, the variables in this research had acceptable 
reliability and validity. 


Table 2. Reliability and validity analysis. 


Mean SD Factor loading CR Cronbach’s a AVE Bl EN ATT PR-~ MI 
Bl 5.21 1.1 0.92 0.96 0.94 0.85 0.92 
EN 4.83 10 0.87 0.94 0.92 0.75 0.61 0.87 
ATT 4.61 1.2 0.73 0.80 0.79 0.69 0.63 0.54 0.83 
PR 4.93 1.1 0.75 0.81 0.70 0.73 0.54 036 0.65 0.85 
MI 4.74 1.4 0.87 0.86 0.77 0.67 0.48 043 064 0.64 0.82 


Note: SD= Standard deviation; CR= Composite reliability 


Table 3. Factor loading analysis of the variables. 


Bl EN ATT PR Ml 
BI 1 0.95 0.56 0.56 0.50 0.43 
BI 2 0.95 0.60 0.60 0.51 0.43 
BI3 0.91 0.59 0.57 0.53 0.46 
ATT1 0.45 0.51 0.76 0.43 0.43 
ATT2 0.50 0.49 0.80 0.37 0.38 
EN2 0.50 0.92 0.50 0.24 0.41 
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Bl EN ATT PR MI 

EN3 0.44 0.89 0.42 0.30 0.32 

EN4 0.68 0.87 0.51 0.44 0.45 

PR1 0.23 -0.01 0.38 0.69 0.22 

PR2 0.23 0.09 0.26 0.70 0.14 

PR3 0.44 0.32 0.45 0.76 0.62 

PR4 0.43 0.37 0.53 0.71 0.49 

MI‘ 0.26 0.14 0.35 0.37 0.75 

MI2 0.53 0.53 0.63 0.70 0.86 

MI3 0.30 0.27 0.51 0.39 0.84 

MI4 0.33 0.29 0.58 0.51 0.86 


Path Coefficient Test of the Structural Model and Prediction Indicator Estimation Model 


PLS was used in this research to analyze the structural model for the correlation strength and direction between 
research variables. If significance appeared after the test, the hypotheses would be supported as expected. PLS can 
be used to analyze and explain the percentage of the variance of exogenous variables to endogenous variables 
and to test the prediction capability of a research model. The structural model was evaluated based on six steps 
proposed by researchers, including: Step 1 - evaluating the multicollinearity of the structural model (VIF); Step 2 - 
setting the significance of the path coefficient as the standard; Step 3 - the test of P= and the path coefficient; Step 
4 - evaluation of the i effect size; Step 5 - evaluation of the predictive relevance (42); and Step 6 - the goodness- 
of-fit index (Hair, Ringle, & Sarstedt, 2011; Reinartz, Haenlein, & Henseler, 2009; Rigdon, 2012; Tenenhaus, Amato, 
& Esposito Vinzi, 2004). First, the variance inflation factor (VIF) was utilized to evaluate the multicollinearity. The 
result showed the maximal VIF of 1.32 for all variables, which conformed to a VIF between 0.2 and 5 as suggested 
by Neter and Wasserman (Hair et al., 2011) and indicated there was no multicollinearity prohlem. 

The structural model was further analyzed using PLS to list the variance explained (R*), standardized path 
coefficient (8), and t value. R* and the path coefficient are the major indicators used to judge a model (Chin, 
1998). The analysis of the structural model is shown in Figure 2. A complete analysis of the model was performed 
in this research. The path coefficient between media interactivity and entertainment was 0.432 (t=4.164) and had 
high significance, thus H, was supported. That is, the higher media interactivity with GAR-STEM apps showed the 
higher entertainment of the apps. The path coefficient between app media interactivity and attitude towards use 
was 0.284 (t=2.961) and had high significance, thus H, was partially supported. The stronger media interactivity 
with GAR-STEM apps showed a higher usage attitude towards using the apps. The path coefficient between app 
media interactivity and practicability was 0.638 (t=9.045) and had high significance, thus H, was supported, reveal- 
ing stronger app media interactivity and higher practicability. The path coefficient between entertainment and 
attitude towards use was 0.293 (t=2.024) and had high significance, thus H, was supported, indicating that the 
higher the entertainment of GAR-STEM apps, the better the attitude towards use. The path coefficient between 
practicability and usage attitude was 0.360 (t=3.374) and had high significance, thus indicating that H, was sup- 
ported. The result also showed a higher practicability of the app and better usage attitude. The path coefficient 
between entertainment and players’ behavioral intention was 0.379 (t=2.949) and had high significance, thus H6 
was supported. That is, higher entertainment will enhance players’ behavioral intention to use GAR-STEM apps. 
The path coefficient between players’ attitude towards use and behavioral intention was 0.275 (t=2.459) and had 
high significance, thus H, was supported. In other words, players with a better attitude towards using an app will 
present higher behavioral intention to use the app. Finally, the path coefficient between the practicability of an app 
and players’ behavioral intention was 0.228 (t=1.692) and had high significance, thus H, was supported, indicating 
the higher the practicability of an app, the higher the behavioral intention of the players. 
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Generally speaking, the R* ofthe endogenous construct ina model can be evaluated as the overall goodness- 
of-fit in PLS (Hulland & Business, 1999). As shown in Figure 2, the R? of the four endogenous constructs were 0.186 
for entertainment, 0.574 for attitude toward use, 0.408 for practicability, and 0.528 for behavioral intention. Attitude 
toward use and behavioral intention showed a high R?, representing a favorable explanation among behavioral 
intention, the four antecedents, and a small unexplained residual. The result indicated a favorable goodness-of-fit 
between behavioral intention and the antecedents. The R? of entertainment was slightly lower, possibly because 
the practicability and usefulness of an app are emphasized in GAR-STEM apps. Entertainment, therefore, is not a 
major component of GAR-STEM apps. In sum, the four endogenous constructs showed a favorable explanation, 
revealing a good explanation of the research model and conformity to the principles suggested by Cohen (1988). 






0.432*** 0.379*** 










0.284*** ATTI1 ee 


H2 (t=2.961) ( 








R’=0.574 \ H8 0.275** 
+SEE 45 | 
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MiI=Media Interactive 
EN=Entertainment 
ATT=Attitude 
PR=Practicability 
Bl=Behavioral Intention 


*; p<0.05, **:p<0.01, ***:p<0.001 


Figure 2. Path coefficient of intention to use GAR-STEM apps. 


In addition to evaluating the predictive power of the model, the = effect size of the structural model was 
another important evaluation indicator. In addition to evaluating the P= of all "endogenous constructs by deleting 
the R- change caused by specific exogenous variables in the model, the = effect size can be used to evaluate 
the effects of the deleted variables on the endogenous variables. Such a measurement is called the effect size. 
The equation is: 





(1) 


1 —fincin ded 


in is generally evaluated by having the values of 0.02, 0.15, and 0.35 represent the «™all, medium, and large 
effect of the exogenous latent variables (Cohen, 1988). Table 4 shows the average of the ia effect size larger than 
0.15 (the medium effect). In addition to R-, researchers should test Stone-Geisser’s Q? value, which is an indicator 
of a model's predictive relevance, to evaluate the prediction accuracy of the correlation pO Geisser, 1974; Stone, 
1974). In regard to Q? in the structural model shown in Table 4, the Q? value of the endogenous latent constructs 
larger than 0 revealed the predictive relevance between the path model and constructs. Tenenhaus et al. (2004) 
and Tenenhaus et al. (2005) proposed the overall goodness-of-fit index (GoF) for PLS as “the possible program to 
verify the overall model of PLS” (Tenenhaus et al., 2004). Wetzels et al. (2009) suggested using the communality 
value of 0.50 (Fornell & Larcker, 1981) and the R’~ value (Cohen, 1988) to evaluate the goodness-of-fit of the PLS 
structural modelasGoF__(0.10),GoF .. (0.25),and GoF,__ ge (0.36). Table 4 shows that the GoF of 0.56 in this research 
was larger than 0.36, thus presenting an excellent apedncse of-fit of the structural model. The model predictive 
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indicators proposed by relative researchers and the goodness-of-fit organized in this research could be used as a 
reference (Table 5) for successive researchers, including the model predictive indicators and the goodness-of-fit 
checklist (Yang & Su, 2017). 


Table 4. Analysis of Model criterion and IPMA. 


Model fit indicator 


(I) (tT) (D) = (l) (1) Performance R? 





MI = 0.43 043 064 - 064 028 035 063 - 0.65 0.65 96.12 - - - 

EN - - - - - - 029 - - 0.38 0.08 0.46 61.86 0.186 - 0.13 

PR - - - - - - 036 - om 0.23 «(0.1 Ges 57.09 0.408 - 0.18 0.56 
ATT - - - - - - - - - - - 0.28 60.55 0.574 0.19 0.26 

BI - - - - - - - - - - - - - 0.528 0.21 0.43 


Note: Ml=Media interactive; EN=entertainment; ATT=Attitude; PR=Practicability; BI=Behavioral Intention 


Table 5. Model predictive indicators and goodness-of-fit checklist. 


Model 


Model evaluation Criterion Suggestion value value Result 
Cronbach's alpha (CA) Values must not be lower than 0.600 CA >0.80 Good 
Composite reliability (CR) Values must not be lower than 0.600. CR >0.80 Good 

; , Values should be significant at the .050 level and 
Indicator loadings higher than 0.70 >0.70 Good 
Internal consistency reliability values ae ghee map ae CR >0.80 Good 
Reliability and Validity Composite Reliability (CR)>=0.70 
& Measurement Model 
- Values must higher than 0.50 
Ena) Average Variance Extracted (AVE) >= 0.500 eve eee 
Cross-loadings Factor Loading should higher than Cross Loading Higher Good 
ee siibare epececon the square root of the ae of each construct should be Higher Good 
higher than the construct’s highest correlation 

Multicollinearity 0.2<VIF< 5 1.32 Good 

Path Coefficient & R* 0.670 (High), 0.333(medium), 0.190(weak). 0.528 Good 

Estimation Model - 
Path coefficients p value <0.05 <0.05 Good 

. 7 0.02~0.15 (weak), 0.15~0.35 (medium), 0.19 ect 

Effect size f~ Value higher than 0.35(strong) 0.21 

si ; n ae a _ - Use blindfolding, Q2 >0 indicate a predictive 0.13 Relevance 

rediction indicator = — Predictive relevance Q- relevance. 

Goodness-of-Fit Index (GoF) 0.1(small), 0.25(medium), 0.36(large) 0.56 Good 
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Result of Importance-Performance Matrix Analysis (IPMA) 


Importance-performance matrix analysis (IPMA) helps to gain additional insights by utilizing the extension 
of the PLS-SEM results and provides additional dimensions (Hock et al., 2010; Kristensen et al., 2000; Slack, 1994; 
Volckner et al., 2010, Su & Cheng, 2019). The analysis is established based on the estimation of the PLS-SEM path 
relevance and by adding the dimension of the latent variable mean value. For the endogenous variables of the 
specific target, IPMA compares the total effect (importance) of the structural model with the mean value of the 
latent variable fraction (performance) to decide the priority of management (or the specific focus in the model). 
More specifically, the result could point out the factors of high importance and relatively worse performance in 
the model for the improvement of successive key design elements and management (Hock et al., 2010; Rigdon, 
Ringle, & Sarstedt, 2010). 

The calculation of importance in IPMA in this research required the total effect of all other constructs (i.e. 
media interactivity, entertainment, attitude towards use, and practicability) on the intention of using the target 
construct (in Figure 2) for judging the importance. The total effect between two constructs was the sum of all direct 
and indirect effects in the structural model and could be represented as total effect=direct effect + indirect effect. 

The construct performance was calculated with: 


(E|x,|—-min |x;]) 
i et, a (2) 
i. I J i 

where Yi; stands for the i-th number (e.g. i=5 represents the latent variable fraction of the fifth observed value 
in the database) of the j-th latent variable (e.g. j=2 represents the second latent variable in the structural model) 
(Anderson & Fornell, 2000; Hock et al., 2010). The IPMA in Table 4 was plotted to show the correlation between the 
importance and performance distribution of the intention to use GAR-STEM apps shown in Figure 3. Furthermore, 
the management meaning could be explained with the acquired regional quadrant. 


Performance 
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Figure 3. Importance and performance distribution of the intention to use GAR-STEM apps. 


Quadrant | showed high importance and high performance and represented the “keep up the good work” 
zone, revealing that players place more stress on entertainment with high satisfaction. In this case, designers must 
maintain the original advantage and enhance the entertainment of digital cultural and creative games. This zone 
therefore was the core competitiveness in the design. Quadrant Il showed low importance but high performance and 
represented the possible overkill zone, showing that players do not focus much on attitude towards use but present 
high satisfaction. Design resources could be adjusted and transferred to other variables with higher importance 
to reduce the cost. Quadrant III showed low importance and low performance and represented the low priority 
zone, revealing that app users do not place high emphasis or satisfaction on practicability. Designers therefore do 
not need to spend too much time on such items. Quadrant IV showed high importance but low performance and 
represented the“concentrate here” zone, presenting the high emphasis but low satisfaction of GAR-STEM app users 
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on media interactivity. In this case, designers should consider improvements and adopt relative design strategies 
and thinking to promote design standards for app media interactivity. Figure 3 reveals that media interactivity was 
located in Quadrant IV (the “concentrate here” zone), with high importance but low performance. In other words, 
the interface and user experience design need to be processed and improved, and further design strategies need 
to be adopted to enhance players’ satisfaction with the media interactivity and encourage players to use the app 
through a favorable interactive interface design. 

The research results revealed that the eight hypotheses were supported and that the path coefficient of each 
hypothesis had high significance (Table 6). After verifying the hypotheses with relevant research, the conclusion 
and suggestions were proposed as follows. 


Table 6. Hypothesis test result. 


H Path Description ee t-Value Results 

H, MI->EN Media interactivity presents a positive effect on entertainment. 0.43 4.164*** Supported 
He MI->ATT — Media interactivity shows a positive effect on attitude towards use. 0.28 2.961*** Supported 
Fs MI-> PR Media interactivity reveals a positive effect on practicability. 0.64 9.045*** Supported 
H, EN-> ATT — Entertainment reveals a positive effect on attitude. 0.29 2.024** Supported 
H. EN-> Bl Entertainment presents a positive effect on behavioral intention. 0.38 2.949*** Supported 
H, PR-> ATT — Practicability presents a positive effect on attitude. 0.36 3.374" Supported 
H, PR-> Bl Practicability shows a positive effect on behavioral intention. 0.28 2.459** Supported 
H, ATT> BI ore use reveals a positive effect on behavioral 0.23 1.692" Siipporied 

Discussion 


Higher Media Interactivity, Expectation of Entertainment, Attitude towards Use, and Practicability 


The results of this research reflected the research viewpoint of Wang, Hsieh, and Song (2012), who indicated 
that users’ perceived media richness of information systems will positively affect the perceived entertainment. 
Sullivan (1999) pointed out interactivity as the dialogue between information systems and users. In addition to 
the interaction, information system users can respond to the information and content offered by information 
systems in the process. Coyle & Thorson (2001) regarded the positive effect of interactivity on the practicability 
of information systems and the assistance in forming consistency between better attitudes towards information 
systems and use behavior. This result was also consistent with the studies of Wang, Hsieh, and Song (2012), Sullivan 
(1999), and Coyle and Thorson (2001). 


Higher Entertainment, Expectation of Attitude towards Use, and Behavioral Intention 


The results reflected the research viewpoint of Hirschman & Holbrook (1982), who pointed out high values 
for entertainment, expectation of attitude towards use, and behavioral intention. Ajzen & Fishbein (1975) claimed 
that attitude and subjective norms are the primary factors in behavioral intention. Elements related to imagination, 
feeling, fun, and symbolic meanings (Hirschman & Holbrook, 1982) or festive and ludic perception (Sherry, 1990) 
often appear with hedonic sensations in the pleasure process. The characteristics of entertainment have often been 
applied in research on behavior (Scarpi, 2005; Sherry, 1990) and used to present subjective and personal contents 
((Babin, Darden, & Griffin, 1994). From the aspect of system use, users perceiving the higher hedonic sensation of 
information systems will reveal higher intention of adoption (Gopal, Bostrom, & Chin, 1992). Such a result was also 
consistent with the research of Scarpi (2005), Sherry (1990), and Gopal, Bostrom, and Chin (1992). 
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Higher Practicability, Expectation of Attitude towards Use, and Behavioral Intention 


Davis (1989) indicated that better system practicability will enhance the expectation of attitude towards use 
and behavioral intention. Meanwhile, Davis (1989) proposed that individual behavioral performance in the Tech- 
nology Acceptance Model is determined by behavioral intention, which can be affected by personal attitudes and 
subjective norms. According to Davis’ Technology Acceptance Model, user attitude towards systems is influenced 
by perceived ease of use and that perceived usefulness and users’ behavioral intention is affected by attitude and 
perceived usefulness, which also affects perceived ease of use (Davis, 1989). Such a model has been broadly applied 
to various studies on technology acceptance behavior, and the hypotheses in the model have been verified several 
times (Ahn et al., 2007; Moon & Kim, 2001; Venkatesh & Bala, 2008). As a result, the research result was consistent 
with Davis (1989), who indicated that system use is decided by behavioral intention, that behavioral intention is 
determined by attitude toward use and perceived usefulness, and that attitude toward use is decided by perceived 
usefulness and perceived ease of use. 


Higher Attitude towards Use results in Higher Expectation of Behavioral Intention 


The research result was consistent with the findings of Davis (1989), who mentioned that good attitude to- 
wards use will result in high behavioral intention. The Technology Acceptance Model was extended from Theory 
of Reasoned Action, which aims to predict and understand human behavior. It is considered that an individual's 
behavior is determined by behavioral intention, while behavioral intention might be influenced by personal attitudes 
and subjective norms. According to Davis’ (1989) Technology Acceptance Model, user attitudes towards systems 
will be affected by perceived ease of use and perceived usefulness, users’ behavioral intention will be influenced 
by attitude and perceived usefulness, and perceived usefulness will affect perceived ease of use. Such a model has 
been broadly applied to research on technology acceptance behavior and the hypotheses in the model have been 
verified numerous times (Ahn et al., 2007; Moon & Kim, 2001; Venkatesh & Bala, 2008). The results were consistent 
with the research of Davis (1989), Venkatesh & Bala (2008), and Moon & Kim (2001). 


Importance-Performance Analysis (IPMA) 


With IPMA, designers can realize the situations of players in the improvement of dimensions and coping 
strategies. Improvement would be required when a factor of the intention to use GAR-STEM apps was emphasized 
but presented low performance. On the contrary, when the importance of a dimension was low but showed high 
performance, it might be overemphasized, and the invested resources could be reduced. Nonetheless, it would 
be meaningless to focus on intention of use with low importance and consider the total effect of the path coef- 
ficient of the structural model, as some the high-importance intention of use items presented good performance. 
Through IPMA, it was found that players highly stress the media interactivity of GAR-STEM apps but do not show 
high satisfaction. The analysis value of media interactivity appeared in Quadrant lV, revealing high importance but 
low performance (the “concentrate here” zone), indicating that players highly emphasize the media interactivity 
interface and user experience design of GAR-STEM apps but are not highly satisfied. In this case, designers should 
take improvement into account and adopt relative design strategies and thinking to promote design standards 
for app media interactivity, such as the experience design of the user interface and players’ experience design, to 
allow players to invest in the situation through favorable interface design. 


Conclusions 


The model proposed in this research could predict the behavioral intention in the application of GAR-STEM. 
In addition, the overall model evaluation scale had good research adaptability. The design of useful STEM teaching 
materials could be used to present and explain concepts in an effective manner and design appropriate teaching 
activities so as to enable all students to understand the contents they have learned. Good STEM teaching material 
design should cover different teaching objectives for different grades, and furthermore should note the under- 
standing level of students on the subject and various teaching strategies and presentation methods in teaching. 
Therefore, this research put forth the following academic and practical implications: 
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Academic and Practical Meaning 


(1) Academic implications: Research on cultural and creative issues is increasing. The prediction model for the 
behavioral intention of GAR-STEM apps proposed in this research presents innovation and offers a reference for 
successful app design. From the viewpoint of the behavioral intention to use GAR-STEM apps, app designs with 
media interactivity, entertainment, and practicability are the key factors in players’ attitude and intention of use. The 
research results therefore could be referenced in regard to GAR-STEM app design and behavioral intention theory. 
(2) Practical implications: In respect of the teaching knowledge in STEM, teachers should be able to organize 
and understand learning contents and present them to students by effective interaction means. Techniques in 
presentation include the application of the most persuasive statements, examples, simulations, explanations, and 
demonstrations using teaching aids to present teaching materials in a way that is acceptable to students. In addi- 
tion, teaching material design through multimedia scenarios could contribute to understanding difficult subjects, 
particularly science, technology, engineering and mathematics. However, teachers should also understand which 
subjects students are interested in, their difficulties in learning, and how to further transfer the contents in teaching 
materials into effective teaching, and they should be able to adjust teaching methods for students with different 
abilities and backgrounds (for example, how to design the experience design and user experience design in rela- 
tion to the media interactive user interface of GAR-STEM). It is important to take the entertainment aspect into 
account. From the perspective of system application, the higher the “entertainment” of an information system a 
user experiences, the stronger the adoption intention will be. 


Research Restrictions and Suggestions for Successive Research 


This research merely utilized scales from the literature for the measurement to understand the inadequacy 
of the theoretical dimensions, and qualitative research based on expert interviews was not taken into consider- 
ation. Meanwhile, the representativeness of the samples was a major restriction in the quantitative research. The 
factors of the behavioral intention of GAR-STEM apps therefore were major concerns. Possible research directions 
are listed below. 

(1) Practical research 

In addition to referring to the literatures, qualitative expert interviews can be included to discuss the key fac- 
tors in the behavioral intention to use GAR-STEM apps so as to reinforce the model prediction. 
(2) Extension of the research model 

Theoretical points of view were adopted in this research; however, only subjective behavioral intention and 
inferences were discussed. Regarding the factors of the behavioral intention to use GAR-STEM apps, it is suggested 
to include expert interviews to make up for the inadequacy of the quantitative research. It is also suggested to 
extend the research model by adding learning style and involvement as moderators in order to observe the mod- 
eration effect of involvement on intention of use. In this case, limited resources could be invested in moderators 
to achieve maximal efficiency. 
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